<div class="tmd-doc">
<p></p>
<h1 class="tmd-header-1">
Some Facts About Go Iterators
</h1>
<p></p>
<div class="tmd-usual">
Contents:
</div>

<ul class="tmd-list tmd-toc">
<li class="tmd-list-item tmd-toc-item"><a href="#introduction">A simple introduction of Go iterators</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#dont-call-yield-if-it-ever-returned-false">Don't call a <code class="tmd-code-span">yield</code> callback again if a call to the callback has ever returned false</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#dont-call-yield-if-it-caller-has-returned">Don't call the <code class="tmd-code-span">yield</code> callback of an iterator function if the iterator function has returned</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#calling-yield-concurrently-needs-synchronizations">Calling a <code class="tmd-code-span">yield</code> callback function concurrently needs careful synchronizations</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#impact-on-goroutine-execution">The top function calls in the call stack of a goroutine will be executed concurrently when ranging over iterator functions</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#equivalence-between-range-over-and-direct-call">Calling an iterator function explicitly might be equivalent to ranging over the iterator function or not</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#design-flaws">The iterator design has a flaw</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#implementation-bugs">The implementation of iterators still has some bugs</a>
</li>
<li class="tmd-list-item tmd-toc-item"><a href="#importance">The importance of iterators has not been proved yet</a>
</li>
</ul>
<p></p>
<h3 id="introduction" class="tmd-header-3">
A simple introduction of Go iterators
</h3>
<p></p>
<p></p>
<div class="tmd-usual">
Go 1.23 inroduced <a href="https://go.dev/blog/range-functions">iterators</a>, Iteators in Go are some functions which can be ranged over. Specifically, function expressions of types whose underlying types are in the following forms can be ranged over.
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">func(func() bool)
func(func(V) bool)
func(func(K, V) bool)

// K and V both denote types.
</code></pre>
<p></p>
<div class="tmd-usual">
All of the types take a callback function parameter which is called <code class="tmd-code-span">yield</code> by convention. Such ballback functions might take zero, one or two parameters and always return a bool result.
</div>
<p></p>
<div id="example-1" class="tmd-usual">
For example, the following Go program prints <code class="tmd-code-span">abc</code> (since Go 1.23).
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"

func main() {
	for v := range abc {
		fmt.Print(v)
	}
}

func abc(yield func(string) bool) {
	if yield("a") {
		if yield("b") {
			yield("c")
		}
	}
}
</code></pre>
<p></p>
<div class="tmd-usual">
The function <code class="tmd-code-span">abc</code> is used as an iterator here. Its parameter <code class="tmd-code-span">yield</code> is a callback function which takes a <code class="tmd-code-span">string</code> parameter and returns a <code class="tmd-code-span">bool</code> result.
</div>
<ul class="tmd-list">
<li class="tmd-list-item">
<div class="tmd-usual">
Three arguments <code class="tmd-code-span">"a"</code>, <code class="tmd-code-span">"b"</code> and <code class="tmd-code-span">"c"</code> are in turn passed to the <code class="tmd-code-span">string</code> paramter and are used as the values of (the instances of) the loop variable <code class="tmd-code-span">v</code>.
</div>
</li>
<li class="tmd-list-item">
<div class="tmd-usual">
The loop body can be basically viewed as the function body of the <code class="tmd-code-span">yield</code> function. Just note that, if the execution of a loop step during ranging over an iterator function terminates the whole loop (such as <code class="tmd-code-span">break</code> the loop), then the corresponding <code class="tmd-code-span">yield</code> call returns false (to the iterator function); otherwise the corresponding <code class="tmd-code-span">yield</code> call returns true.
</div>
</li>
</ul>
<p></p>
<div class="tmd-usual">
The iterator function in the above example can also be written as
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">func abc(yield func(string) bool) {
	_ = yield("a") &amp;&amp; yield("b") &amp;&amp; yield("c")
}
</code></pre>
<p></p>
<h3 id="dont-call-yield-if-it-ever-returned-false" class="tmd-header-3">
Don't call a <code class="tmd-code-span">yield</code> callback again if a call to the callback has ever returned false
</h3>
<p></p>
<div class="tmd-usual">
The callback function of an iterator function must not be called any more if a call of the callback function ever returned false. For example, the following program crashes when running.
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"

func main() {
	for v := range ab {
		fmt.Println(v)
		break // remove this line to avoid crashing.
	}
}

func ab(yield func(string) bool) {
	var done = yield("a")
	fmt.Println(done) // false
	
	yield("b") // runtime error: range function
	           // continued iteration after function
	           // for loop body returned false.
}
</code></pre>
<p></p>
<div class="tmd-usual">
The official Go compiler doesn't check such invalid uses. So such an invalid use produces a runtime (recoverable) error.
</div>
<p></p>
<div class="tmd-usual">
Note, <code class="tmd-code-span">go vet</code> also doesn't check such invalid uses (up to v1.24.x).
</div>
<p></p>
<h3 id="dont-call-yield-if-it-caller-has-returned" class="tmd-header-3">
Don't call the <code class="tmd-code-span">yield</code> callback of an iterator function if the iterator function has returned
</h3>
<p></p>
<div class="tmd-usual">
The callback function of an iterator function must not be called any more if the iterator function has returned. For example, the following program crashes when running.
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"

var f func(string) bool

func main() {
	for v := range abc {
		fmt.Print(v)
	}
	f("d") // runtime error: range function
	       // continued iteration after
	       // whole loop exit
}

func abc(yield func(string) bool) {
	_ = yield("a") &amp;&amp; yield("b") &amp;&amp; yield("c")
	f = yield
}
</code></pre>
<p></p>
<h3 id="calling-yield-concurrently-needs-synchronizations" class="tmd-header-3">
Calling a <code class="tmd-code-span">yield</code> callback function concurrently needs careful synchronizations
</h3>
<p></p>
<div class="tmd-usual">
<a href="https://github.com/golang/go/issues/68897">Parallel calls to a <code class="tmd-code-span">yield</code> callback function are not synchronized</a> by default. You need to do the synchronizations by yourself.
</div>
<p></p>
<p></p>
<div class="tmd-usual">
For example, the following program crashes for a confusing panic.
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"
import "sync"

var c = make(chan int)

func main() {
	// runtime error: range function continued iteration after loop body panic
	for v := range oneHundred {
		&lt;-c
		fmt.Println(v)
	}
}

func oneHundred(yield func(int) bool) {
	var wg sync.WaitGroup
	for v := range 100 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			yield(v)
		}()
	}
	close(c)
	wg.Wait()
}
</code></pre>
<p></p>
<div class="tmd-usual">
To make it work, it should be synchronized, like
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">func abc(yield func(int) bool) {
	var wg sync.WaitGroup
	var mu sync.Mutex // use a mutex to sync
	for v := range 100 {
		wg.Add(1)
		go func() {
			defer wg.Done()
			mu.Lock()         // prevent yield being
			defer mu.Unlock() // called parallelly.
			yield(v)
		}()
	}
	close(c)
	wg.Wait()
}
</code></pre>
<p></p>
<div class="tmd-usual">
However, it is strongly recommended not to use <code class="tmd-code-span">yield</code> callback functions in other goroutines.
</div>
<p></p>
<h3 id="impact-on-goroutine-execution" class="tmd-header-3">
The top function calls in the call stack of a goroutine will be executed concurrently when ranging over iterator functions
</h3>
<p></p>
<div class="tmd-usual">
When an iterator function is ranged over, it is called implicitly. During ranging over the iterator function, the execution of a goroutine will jump back and forth between the iterator function call and the loop body (the loop body is a statement within the caller of the iterator function call).
</div>
<p></p>
<div class="tmd-usual">
For example, the following program prints <code class="tmd-code-span">0a1b2c</code>.
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"

func main() {
	for v := range abc {
		fmt.Print(v)
	}
}

func abc(yield func(string) bool) {
	for i, v := range "abc" {
		fmt.Print(i)
		if (!yield(string(v))) {
			break
		}
	}
}
</code></pre>
<p></p>
<div class="tmd-usual">
The following is another example, in which the execution of a goroutine will jump back and forth between 2<sup>22</sup> iterator function calls and the loop body in the <code class="tmd-code-span">main</code> function. Finally, the program prints <code class="tmd-code-span">4194304</code>, <code class="tmd-code-span">4194305</code> and <code class="tmd-code-span">4194306</code> in turn.
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">package main

func wrap(it func(yield func(int) bool)) func(yield func(int) bool) {
	return func(yield func(int) bool) {
		for v := range it {
			yield(v+1)
		}
	}
}

func main() {
	iterator := func(yield func(int) bool) {
		_ = yield(0) &amp;&amp; yield(1) &amp;&amp; yield (2)
	}
	for range 1 &lt;&lt; 22 { // 23 crashes the program
		iterator = wrap(iterator)
	}
	for v := range iterator {
		println(v)
	}
}
</code></pre>
<p></p>
<h3 id="equivalence-between-range-over-and-direct-call" class="tmd-header-3">
Calling an iterator function explicitly might be equivalent to ranging over the iterator function or not
</h3>
<p></p>
<div class="tmd-usual">
For the iterator function shown above:
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">func abc(yield func(string) bool) {
	_ = yield("a") &amp;&amp; yield("b") &amp;&amp; yield("c")
}
</code></pre>
<p></p>
<div class="tmd-usual">
the following two ways of using it are equvalent.
</div>
<p></p>
<div class="tmd-tab">
<input type="radio" class="tmd-tab-radio" name="tmd-tab-0" id="tmd-tab-0-input-1" checked>
<label for="tmd-tab-0-input-1" class="tmd-tab-header tmd-tab-label">
Way 1
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">	for v := range abc {
		fmt.Print(v)
	}
</code></pre>
<p></p>
</div>
<input type="radio" class="tmd-tab-radio" name="tmd-tab-0" id="tmd-tab-0-input-2">
<label for="tmd-tab-0-input-2" class="tmd-tab-header tmd-tab-label">
Way 2
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">	abc(func(v string) bool {
		fmt.Print(v)
		return true
	})
</code></pre>
</div>
</div>
<p></p>
<div class="tmd-usual">
However, for the following iterator function:
</div>
<p></p>
<pre class="tmd-code">
<code class="language-Go">func abcThenPanic(yield func(string) bool) {
	_ = yield("a") &amp;&amp; yield("b") &amp;&amp; yield("c")
	panic("bye")
}
</code></pre>
<p></p>
<div class="tmd-usual">
the following two ways of using it are not equvalent.
</div>
<p></p>
<div class="tmd-tab">
<input type="radio" class="tmd-tab-radio" name="tmd-tab-1" id="tmd-tab-1-input-1" checked>
<label for="tmd-tab-1-input-1" class="tmd-tab-header tmd-tab-label">
Way 1
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">	for v := range abcThenPanic {
		defer func() {
			recover()
		}()
		fmt.Print(v)
	}
</code></pre>
<p></p>
</div>
<input type="radio" class="tmd-tab-radio" name="tmd-tab-1" id="tmd-tab-1-input-2">
<label for="tmd-tab-1-input-2" class="tmd-tab-header tmd-tab-label">
Way 2
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">	abcThenPanic(func(v string) bool {
		defer func() {
			recover()
		}()
		fmt.Print(v)
		return true
	})
</code></pre>
</div>
</div>
<p></p>
<div class="tmd-usual">
When loop bodies contain <code class="tmd-code-span">goto</code> or <code class="tmd-code-span">return</code> statements, the equivalence between the two ways becomes even less likely.
</div>
<p></p>
<h3 id="design-flaws" class="tmd-header-3">
The iterator design has a flaw
</h3>
<p></p>
<div class="tmd-usual">
The following two programs should behave the same, but they don't now (as of Go toolchain v1.24.1):
</div>
<p></p>
<div class="tmd-tab">
<input type="radio" class="tmd-tab-radio" name="tmd-tab-2" id="tmd-tab-2-input-1" checked>
<label for="tmd-tab-2-input-1" class="tmd-tab-header tmd-tab-label">
Program 1
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">package main

func main() {
	defer func() {
		var v = recover()
		println(v == 123)
	}()
	for range iter {
		panic(123)
	}
}

func iter(yield func() bool) {
	defer func() {
		recover()
	}()
	yield()
}
</code></pre>
<p />
<div class="tmd-usual">
The program prints <code class="tmd-code-span">false</code>.
</div>
<p></p>
</div>
<input type="radio" class="tmd-tab-radio" name="tmd-tab-2" id="tmd-tab-2-input-2">
<label for="tmd-tab-2-input-2" class="tmd-tab-header tmd-tab-label">
Program 2
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">package main

func main() {
	defer func() {
		var v = recover()
		println(v == 123)
	}()
	for range iter {}
	panic(123)
}

func iter(yield func() bool) {
	defer func() {
		recover()
	}()
	yield()
}
</code></pre>
<p />
<div class="tmd-usual">
The program prints <code class="tmd-code-span">true</code>.
</div>
</div>
</div>
<p></p>
<div class="tmd-usual">
The cause of the behavior difference is, by the current design,
</div>
<ol class="tmd-list">
<li class="tmd-list-item">
<div class="tmd-usual">
<a href="https://github.com/golang/go/issues/71830">the panic created in the loop body of ranging over an iterator function will propagate to the iterator function</a>.
</div>
</li>
<li class="tmd-list-item">
<div class="tmd-usual">
the iterator function should not recover the panic created in the loop body. If the iterator function recovers the panic propagated from the loop body, then the runtime will create a new runtime panic implicitly.
</div>
</li>
</ol>
<p></p>
<p></p>
<div class="tmd-usual">
If the yield call just returns false when the loop body creates a panic, then there will be no behavior differences between the two programs.
</div>
<p></p>
<h3 id="implementation-bugs" class="tmd-header-3">
The implementation of iterators still has some bugs
</h3>
<p></p>
<div class="tmd-usual">
For example, the following two programs should have the same behavior, but they don't now (as of Go toolchain v1.24.1).
</div>
<p></p>
<div class="tmd-tab">
<input type="radio" class="tmd-tab-radio" name="tmd-tab-3" id="tmd-tab-3-input-1" checked>
<label for="tmd-tab-3-input-1" class="tmd-tab-header tmd-tab-label">
Program 1
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"

func main() {
	defer foo()
	panic(123)
}

func foo() {
	for range iter {
		fmt.Println(recover())
	}
}

func iter(yield func() bool) {
	yield()
}
</code></pre>
<p />
<div class="tmd-usual">
The program prints <code class="tmd-code-span">&lt;nil&gt;</code> then crashes.
</div>
<p></p>
</div>
<input type="radio" class="tmd-tab-radio" name="tmd-tab-3" id="tmd-tab-3-input-2">
<label for="tmd-tab-3-input-2" class="tmd-tab-header tmd-tab-label">
Program 2
</label>
<div class="tmd-tab-content">
<pre class="tmd-code">
<code class="language-Go">package main

import "fmt"

func main() {
	defer foo()
	panic(123)
}

func foo() {
	for range 1 {
		fmt.Println(recover())
	}
}
</code></pre>
<p />
<div class="tmd-usual">
The program prints <code class="tmd-code-span">123</code> then exits normally.
</div>
</div>
</div>
<p></p>
<h3 id="importance" class="tmd-header-3">
The importance of iterators has not been proved yet
</h3>
<p></p>
<div class="tmd-usual">
Iterators surely have certain usefulness. However, it is still not proved that it is worth adding so much complexity to the language.
</div>
<p></p>
</div>
